1. Field of the Invention
The present invention relates to a camera shake correction technique.
2. Description of the Related Art
A recent increase in the resolution and zooming ratio of an image sensing device has posed a problem of blur of a sensed image caused by camera shake upon image sensing, and conventionally led to wide use of image sensing devices with a camera shake correction function. Such an image sensing device with a camera shake correction function generally uses an optical camera shake correction scheme of optically correcting a camera shake using a sensor configured to detect a shift vector generated by a camera shake and a driving device configured to control the relative positions of the lens and an image sensor so as to cancel the camera shake.
On the other hand, a method of correcting a camera shake by image processing has also been examined. To correct a camera shake for one blurred image, a method is known which, interpreting the process of image degradation caused by a camera shake as convolution by a filter, calculates a filter based on the speed of camera shake, and removes the camera shake by deconvolution (Japanese Patent Laid-Open Nos. 2006-279807 and 2004-205799).
Generally, the frequency characteristic of a shake filter has the property of decreasing to zero at a certain frequency. This poses a problem that the information of the frequency at which the frequency characteristic becomes zero is lost. To prevent this, there is proposed a technique of correcting a camera shake or object blur by opening/closing the shutter at random during exposure when acquiring an image and performing a deconvolution operation using the shutter open/close information (Japanese Patent Laid-Open No. 2008-310797). This technique is called Coded Exposure. Using a favorable Coded Exposure pattern enables to prevent the frequency characteristic of a shake filter from decreasing to zero and implement shake correction.
However, the methods disclosed in Japanese Patent Laid-Open Nos. 2006-279807 and 2004-205799 have the following problem. First, the methods disclosed in Japanese Patent Laid-Open Nos. 2006-279807 and 2004-205799 do not control the exposure condition such as shutter open/close. For this reason, the frequency characteristic of a shake filter at a specific frequency decreases to zero (this phenomenon will be referred to as a zero power frequency hereinafter), and simple deconvolution is impossible. To avoid this problem, Japanese Patent Laid-Open No. 2006-279807 uses a pseudo inverse matrix, and Japanese Patent Laid-Open No. 2004-205799 uses a Wiener filter. However, it is not possible to restore the information of the frequency at which a zero power frequency has occurred in either case.
On the other hand, the method of Japanese Patent Laid-Open No. 2008-310797 uses Coded Exposure to avoid the problem of zero power frequency as in the Japanese Patent Laid-Open Nos. 2006-279807 and 2004-205799. However, the method is not applicable to a camera shake of non-uniform motion.